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I. 


INTRODUCTION 


The Department of Defense (DOD) expects to invest $1.5 trillion in defense 
acquisition programs (GAO, 2015). This does not include the multitude of other 
acquisitions that the federal government procures on a yearly basis. In the current 
budgetary environment, it has become increasingly important to make best use of scarce 
funding and obtain the best value for every good or service. The method to steer the 
acquisition workforce into properly buying goods and services is the contract 
management process. Rendon and Snider state, “the contract-management process is 
typically used as the vehicle for progressing through the defense acquisition life cycle” 
(Rendon & Snider, 2008, p. 165). The six contract management process steps are 
“procurement planning, solicitation planning, solicitation, source selection, contract 
administration, and contract closeout” (Rendon & Snider, 2008, p. 165). 

The source selection step of the contract management process is vital to executing 
the appropriate contracting award strategy to deliver the best value to the government. In 
order to achieve best value for all stakeholders, government contracting officers have two 
primary source selection strategies at their disposal: lowest price technically acceptable 
(LPTA) and tradeoff. The LPTA source selection strategy is “appropriate when best 
value is expected to result from selection of the technically acceptable proposal with the 
lowest evaluated price” (FAR, 2016, 15.101-2(a)). The FAR also explains that tradeoff 
“is appropriate when it may be in the best interest of the Government to consider award 
to other than the lowest priced offeror or other than the highest technically rated offeror” 
(FAR, 2016, 15-101-1(a)). The tradeoff strategy incorporates factors besides cost or 
price in determining if the good or service best meets the need of the government. 
Examples of these factors include past performance, technical ability, and sub-contractor 
management—any factor that is unrelated to cost or price. 
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A. 


PURPOSE 


This research analyzes LPTA and tradeoff source selection strategies and contract 
outcomes to determine if a relationship exists. Earned value management (EVM) metrics 
(when available), Contractor Performance Assessment Reporting System (CPARS) 
ratings, and procurement administrative lead-time (PALT) are examples of contract 
outcomes that the research team will analyze. This research is part of an ongoing research 
stream guided by Dr. Rene Rendon and Major Karen Landale of the Graduate School of 
Business and Public Policy at the Naval Postgraduate School. Our research incorporates 
new data extracted from a large sample of contracts and associated files at the Space and 
Naval Warfare Systems Command (SPAWAR) and Naval Supply Systems Command 
(NAVSUP). 

B. RESEARCH QUESTIONS 

The following questions will be addressed in our research: 

1. Are pre-award contract elements affected by source selection strategy? 
Pre-award elements include amendments to solicitation, how long it takes 
to award the contract (PALT), contract type, and whether there was a 
protest. 

2. Are post-award contract elements affected by source selection strategy? 
Post-award elements include CPARS ratings, and, if applicable, EVM 
metrics. 

3. Are there any patterns or trends based on federal supply codes when 
LPTA or tradeoff is used as a source selection strategy at different systems 
commands? 

C. METHODOLOGY 

The methodology for this research consists of a review of applicable literature and 
a post-collection data analysis. The research team reviewed completed contract files and 
other contract-related documents from the Space and Naval Warfare Systems Command 
(SPAWAR) and the Naval Supply System Command (NAVSUP). The team attempted to 
gather an equal mix of LPTA and tradeoff contract files to increase the ongoing research 
sample size. The research team examined each of the LPTA and tradeoff contract files 
and gathered the required data elements reflected in the Appendix. Statistical analysis, 
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including analysis of covariance (ANCOVA), was used to identify possible relationships 
between source selection strategy and contract outcomes. 

D. RESEARCH BENEFITS 

This research will help government contracting agencies identify the proper 
source selection strategy that is most advantageous to the government. The government 
contracting agency will better analyze the pre and post-award contract elements and 
have the knowledge to decide the best contracting source selection strategy for a 
best value contract outcome. The contracting agency will understand when the 
tradeoff source selection strategy is worth the added cost for the respective requirement. 
Finally, this research taps into two major system commands not previously used in this 
ongoing research stream. It will help decide if the joint data in the ongoing research have 
any trends and if these trends can be applied for best practice government contracting 
agency use. 

E. RESEARCH LIMITATIONS 

Our main constraint was having enough time to gather sufficient data to 
incorporate into the ongoing research stream. Due to time constraints, we only had five 
business days to gather the contract data elements between two SPAWAR activities and 
one NAVSUP activity. The research team’s goal was to view 50 separate contract files 
and populate the table in the Appendix with the required data elements. 

We reviewed 76 contract files, but we could only extract all required data from 
50 of them. Twenty-six of the contract files were retrieved from an electronic database 
and did not include all data elements for our research. The research team did not have 
access to the electronic repository of contract files and had to rely on site support for 
the required information. This proved to be a lengthy process, and given the time 
considerations, the research team could not gather the required complete information. 

Another challenge we encountered was achieving an acceptable learning curve for 
data extraction at each site location. Although the hard copy contract files at each 
location were similar, the organization of the contract files between locations differed 
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slightly. Once we became proficient at data collection within one location, the research 
team had limited time before moving on to the next location. 

The final challenge was gathering the required data elements for the contract files 
based on the individual site location. For example, NAVSUP only had limited contracts 
containing CPARS information. However, SPAWAR had ah required CPARS 
information, but due to the electronic filing system, it was difficult to find other required 
data elements to fully populate the table of data elements. 

F. REPORT ORGANIZATION 

The organization of this report is subsequently described. In Chapter I, the 
introduction comprises the research’s purpose, the research questions, the methodology, 
and the benefits and research limitations. Chapter II comprises a review of applicable 
literature of the government contract management process, the LPT A and tradeoff source 
selection strategies, and the current debate of the appropriate use of LPT A and tradeoff 
strategies. Chapter III entails an overview of the SPAWAR and NAVSUP organizations. 
Chapter IV submits the data analysis results from SPAWAR and NAVSUP and provides 
implications for DOD contract management. Chapter V summarizes and concludes the 
report with answers to the research questions and recommended areas for continued 
research. 

G. SUMMARY 

Chapter I provided the research purpose, questions for the research, the 
methodology, benefits and limitations, and report organization. Chapter II will look at the 
applicable literature related to the government contract management process, LPT A and 
tradeoff source selection strategies, and the current debate on the applicable use of LPTA 
and tradeoff source selection strategies. 
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II. LITERATURE REVIEW 


The focus of the literature review is to gamer a thorough understanding of 
applicable contract management statutes and regulations. Additionally, the contract 
management process (including the three phases and six steps) as described by Garrett, 
and the two primary source selection strategies (LPTA and tradeoff) as described in the 
FAR are discussed. Lastly, the literature review will address the current debate with 
respect to the proper use of LPTA and tradeoff source selection strategies. 

A. CONTRACT MANAGEMENT STATUTES AND REGULATIONS 

In the contract management process, a contract, as defined by the FAR 2.101(b) 
states that it is 

A mutually binding legal relationship obligating the seller to furnish the 
supplies or services (including construction) and the buyer to pay for 
them. It includes all types of commitments that obligate the Government 
to an expenditure of appropriated funds and that, except as otherwise 
authorized, are in writing. In addition to bilateral instruments, contracts 
include (but are not limited to) awards and notices of awards; job orders or 
task letters issued under basic ordering agreements; letter contracts; 
orders, such as purchase orders, under which the contract becomes 
effective by written acceptance or performance; and bilateral contract 
modifications. Contracts do not include grants and cooperative agreements 
covered by 31 U.S.C.6301, et seq. 

As a more practical definition, Garrett defines a contract as “a relationship 
between two parties, such as a buyer and seller, that is defined by an agreement about 
their respective rights and responsibilities” (Garrett, 2010, p. 416). 

The following laws and regulation govern the contract management process: 

• Small Business Act of 1953 (SBA) 

• Truthful Cost or Pricing Data Act (41 U.S.C. 35) 

• Competition in Contracting Act of 1984 (CICA) 

• Federal Acquisition Streamlining Act of 1994 (FASA) 

• Federal Acquisition Refonn Act of 1996 (FARA) 

• Federal Acquisition Regulation (FAR) 
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1 . 


SBA of 1953 


Due to concerns over a shrinking industrial base and a lack of small business 
representation in government acquisition. Congress passed the SBA of 1953 mandating 
the government 

aid, counsel, assist, and protect, insofar as is possible, the interests of 
small-business concerns in order to preserve free competitive enterprise, to 
insure that a fair proportion of the total purchased and contracts or 
subcontracts for property and services for the Government (including but 
not limited to contracts or subcontracts for maintenance, repair, and 
construction) be placed with small business enterprises, to insure that a 
fair proportion of the total sales of Government property be made to such 
enterprises, and to maintain and strengthen the overall economy of the 
Nation. 

The SBA of 1953 defines a small business as “one which is independently owned 
and operated and which is not dominant in its field of operation” (Small Business Act, 
1953). 


2. Truthful Cost or Pricing Data Act 

Once referred to as the Truth in Negotiations Act (TINA), TINA has been 
renamed the Truthful Cost or Pricing Data Act (FAR, 1.110). The Procedures, Guidance, 
and Infonnation addendum to the Defense Federal Acquisition Regulation Supplement 
states that 

(1) Contracting officers must purchase supplies and services from 
responsible sources at fair and reasonable prices. The Truth in 
Negotiations Act (TINA) (10 U.S.C. 2306a and 41 U.S.C. chapter 35) 
requires offerors to submit certified cost or pricing data if a procurement 
exceeds the TINA threshold and none of the exceptions to certified cost or 
pricing data requirements applies. Under TINA, the contracting officer 
obtains accurate, complete, and current data from offerors to establish a 
fair and reasonable price (see FAR 15.403). TINA also allows for a price 
adjustment remedy if it is later found that a contractor did not provide 
accurate, complete, and current data. 

(2) When certified cost or pricing data are not required, and the 
contracting officer does not have sufficient data to determine price 
reasonableness, FAR 15.402(a)(2) requires the offeror to provide whatever 
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data the contracting officer needs in order to detennine fair and reasonable 
prices. 

(3) Obtaining sufficient data from the offeror is particularly critical in 
situations where an item is detennined to be a commercial item in 
accordance with FAR 2.101 and the contract is being awarded on a sole 
source basis. This includes commercial sales data of items sold in similar 
quantities and, if such data is insufficient, cost data to support the 
proposed price. 

3. CICA of 1984 

CICA of 1984 mandates the maximization of full and open competition, whenever 
feasible, for awarding contracts (Nash, Schooner, Obrien-DeBakey, & Edwards, 2007, 
p. 111). The contracting officer is required “to execute a justification—and obtain 
approval of it—for any procurement in which full and open completion would not be 
obtained” (Nash et ah, 2007, p. 111). Contracting officers may bypass competition and 
award sole-source in the following situations: 

• “Only one responsible source and no other supplies or services will satisfy 
agency requirements” (FAR 6.302-1) 

• “Unusual and compelling urgency” (FAR 6.302-2) 

• “Industrial mobilization; engineering, developmental, or research 

capability; or expert services” (FAR 6.302-3) 

• “International agreement” (FAR 6.302-4) 

• “Authorized or required by statute” (FAR 6.302-5) 

• “National security” (FAR 6.302-6) 

• “Public interest” (FAR 6.302-7) 

4. FASA and FARA 

In recognition of the innovations generated in private industry best practices, 

Congress enacted FASA to urge the federal government to operate more in line 

with the commercial marketplace and to buy commercial items whenever feasible 

(OUSD[AT&L], 2011, p. iv). As a mechanism to facilitate this, the FASA raised the 

threshold for simplified acquisition procedures to $100,000 (Nash et ah, 2007, p. 254), 

which was subsequently raised again to $150,000 (FAR 2.101). The Office of the 

Undersecretary of Defense for Acquisition, Technology, and Logistics (OUSD[AT&L] 

requires that “all of acquisition must move to a price-based, market-driven environment 

from requirements development through properly disposal. Source selection must be 
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made on a ‘best value’ not ‘cheapest price’ basis” (OUSD[AT&L], 2011, p. iv). In 
addition, FASA made market research and past performance monitoring and 
documentation a statutory requirement and created an allowance for best value. The 
FARA additions are detailed in Figure 1. 


• Expanded definition of “commercial item" and its applicability to 
include: 

- items which have evolved from commercial items 

- items that are commercial with modifications to meet 
Government unique requirements 

- combinations of commercial items and services for Government 
use 

- non-developmental items (NDI - items originally developed 
and/or sourced by a Government agency) 

- services at catalog or market prices 

• Prohibited the use of certified cost and pricing data under TINA 
for commercial items. 

• Allows the utilization of Simplified Acquisition Procedures (SAP) 
up to $5 million for commercial goods and services. 


Figure 1. FARA Additions to FASA. Source: Yoder (2007). 


The OUSD[AT&L] Commercial Item Fiandbook (Vers. 2.0) states that “the 1996 
Federal Acquisition Reform Act continued the intent of FASA, creating opportunities to 
improve procedures, promote completion, and purchase commercial items with the ease 
of non-governmental agencies” (OUSD[AT&L], 2011, p. iv). 

5. FAR 

The FAR regulates how the federal government acquires supplies and services; 
the set of rules “is prepared, issued, and maintained, and the FAR System is prescribed 
jointly by the Secretary of Defense, the Administrator of General Services, and the 
Administrator, National Aeronautics and Space Administration, under their several 
statutory authorities” (FAR 1.103b). Many federal agencies and departments maintain 
their own supplements to the FAR including: 

• Defense Federal Acquisition Regulation Supplement (DFARS) 

• Navy and Marine Corps Acquisition Supplement (NMCARS) 


8 



• Army Federal Acquisitions Regulation Supplement (AFARS) 

• Air Force Federal Acquisition Regulation Supplement (AFFARS) 

• Homeland Security Acquisition Regulation (HSAR) 

Thus far, the management of government contracts through statutes and 
regulations has been the focus. The contract management process will be discussed in the 
following section. 

B. CONTRACT MANAGEMENT PROCESS 

This section examines the contract management process in terms of the three 
phases and six steps that make up the process. There are two vantage points from which 
to view contracts: the seller and the buyer. The seller provides services or finished goods, 
such as weapons systems, for payment. When dealing with a DOD acquisition, the seller 
is most often a defense contractor. The buyer, who is the government, in this case, 
purchases these goods and services using contracts. Because contracts can be complex in 
nature, contract management is often viewed as “the art and science of managing a 
contractual agreement throughout the contracting process” (Garrett, 2010, p. 18). 
Contractors often fulfill requirements in support of key operations because the DOD is 
not adequately manned to provide all required products and services in-house. Contracts 
are created, as the principal means to make sure that the government and contractor 
understand their individual responsibilities within the agreement. These documents make 
the agreement between buyer and seller legally binding, thereby reducing any uncertainty 
or risk that the two parties will fail to meet their obligations as set forth in the contract. 
However, successful contract outcomes require an ethical business relationship between 
the government and contractor(s) in addition to strict adherence to contract requirements. 

Next, we turn to the three phases, and the six steps contained within those phases, 
of the contract management process, as detailed by Garrett (2010). Both the buyer and the 
seller conduct six steps as part of the contract management process. However, the six 
steps are not the same between the two parties. This research focuses on the buyer’s 
steps. Figure 2 depicts the contract management process. 
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Figure 2. The Three-Phase Contract Management Process. 

Source: Garrett (2010). 


1. Pre-award Phase 

The pre-award phase of the contract management process includes procurement 
planning, solicitation planning, and solicitation (Garrett, 2010, p. 20). FAR 7.102 states 
“agencies will perform acquisition planning and conduct market research for all 
acquisitions.” The acquisition team uses market research to determine existing 
capabilities available within industry to meet the requirements that the government 
chooses not fulfdl in-house as part of the make-or-buy decision (Garrett, 2010, p. 23). 

a. Step 1: Procurement Planning 

Procurement planning can be recognized as “the process of identifying which 
business needs can be best met by procuring products or services outside the 
organization. This process involves determining whether to procure, what to procure, 
how to procure, and when to procure” (Garrett, 2010, p. 81). Procurement planning 
encompasses many elements. These elements include market research, source selection 
strategy, and contract type selection. Market research helps determine whether industry 
has the ability to handle the requirement, whether going forward with the solicitation will 
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be cost effective, and whether the best source selection strategy has been identified. 
Market research may be done through several means. Some examples are site visits, pre¬ 
solicitation conferences, literature published by providers, review of previously awarded 
contracts, and other open-source information commonly available on the Internet. During 
this step, a risk analysis must also be conducted. This is the process of determining how 
risk can affect “cost, schedule, and performance objectives of the project” (Rendon & 
Snider, 2008, p. 128). A guiding document, called the procurement management plan, is 
then created in order to provide a description of how to manage the rest of the 
procurement process of a contract. The statement of work (SOW) and additional 
preliminary documents will be generated to use in the solicitation planning step. 

b. Step 2: Solicitation Planning 

Next in the pre-award phase comes the solicitation planning step. A major 
element of solicitation planning is “preparing the documents needed to support the 
solicitation” (Garrett, 2010, p. 88). The SOW is considered a key ingredient of 
solicitation planning (Garrett, 2010, p. 88). The government generates a SOW to 
explicitly describe the requirements that the contractor must produce or perform. As a 
result, requirements must be clearly articulated within the SOW to avoid 
misinterpretation. More specifically, if the SOW is not an accurate portrayal of the 
requirements, then the administration process could become unnecessarily (and 
unjustifiably) problematic because its deficiencies will be passed on to the remaining 
steps of the contracting process. However, a statement of objectives (SOO) and 
performance work statement (PWS) may be utilized if appropriate. These allow for a 
less-intrusive management style by the government because the contractor is allowed to 
determine the method for fulfilling the requirement. Essentially, this document describes 
the product or service that the government requires and allows the contractor to find a 
way to accomplish the end state. 

A hierarchy of priorities can be fonned once the proposal evaluation factors are 
developed in the process. According to FAR 15.305, these factors may include “cost or 
price evaluation,” “past performance evaluation,” “technical evaluation,” “cost 
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information,” and “small business contracting evaluation” (FAR 15.305a 1-5). When 
conducting solicitation planning, the government must detennine whether to use one of 
two methods: request for proposal (RFP) or invitation for bid (IFB). If the government 
determines that an IFB will be used, then the government must use an LPTA source 
selection strategy and the “bids shall be evaluated without discussions” (FAR 14.10Id). 
Conversely, if the government decides to issue an RFP, an LPTA or tradeoff source 
selection strategy may be used with or without discussions. The SOW, SOO, or PWS and 
other appropriate documentation will be included as part of the solicitation package. 

Section M of the contract file is an important part of the solicitation planning step 
because it lays out the evaluation criteria and their relative importance (FAR 15.204-5b). 
As a result, close consideration of this section’s content must be observed in the 
solicitation planning step. Section M will also state the source selection strategy (LPTA 
or tradeoff). As part of a tradeoff source selection strategy, the solicitation shall state, 
“whether all evaluation factors other than cost or price, when combined, are 

1. Significantly more important than cost or price; 

2. Approximately equal to cost or price; or 

3. Significantly less important than cost or price (10 U.S.C. 

2305(a)(3)(A)(iii) and 41 U.S.C. 3306(c)(1)(C))” (FAR 15.304e). 

However, if an LPTA source selection strategy is used, “price is the detennining 
factor for award”assuming the contractor meets the minimum technical specifications and 
past performance requirements (Bunting, 2014, II.C.3). As a result, the contractor 
offering the highest technical specifications or past perfonnance will not receive a 
contract award if they are not the lowest technically acceptable offeror. 

c. Step 3: Solicitation 

Solicitation is defined as “any request to submit offers or quotations to the 
Government” (FAR, 2016, 2.101). Part of solicitation is making sure “that all sellers have 
a clear, common understanding of the procurement (both technical requirements and 
contract requirements)” (Garrett, 2010, p. 91). This can be done through bidders’ 
conferences, which can be done in person or by digital means (Garrett, 2010, p. 91). In 
addition to known qualified sellers, additional sellers can be found in this step through 
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advertising (some types of procurement may make finding additional sellers mandatory). 
The government advertises through a government-wide point of entry (GPE). It is “the 
one point of entry to be designated by the Administrator of OFPP that will allow the 
private sector to electronically access procurement opportunities Government wide” 
(FAR, 2016, 2.101). Otherwise, normal advertising outlets can be used. Ultimately, the 
solicitation step culminates in fonnal bids or proposals submitted by interested 
contractors. 

2. Award Phase 

The second of the three phases is the award phase, which contains the source 
selection step. In this phase, the prospective contractor is awarded the contract. Before a 
contract award/purchase can be made, the contractor must be verified to be responsible in 
accordance with FAR 9.103. Both parties may engage in contract negotiations at this 
point to set tenns and conditions that will provide a mutually beneficial environment for 
attaining their respective goals, or achieving a “win-win” agreement (Garrett, 2010, 
p. 136). 


a. Step 4: Source Selection 

The FAR states that “the objective of source selection is to select the proposal that 
represents the best value” to the government and the taxpayer (FAR, 2016, 15.302). The 
nature of the procurement often influences the execution of the source selection strategy. 
Depending on whether a sealed bid or a proposal is requested, the process will vary. 
Sealed bids simply use price as the deciding factor, as long as the bid is in accordance 
with the minimum technical and past performance criteria laid out in the IFB. Contracts 
awarded using a tradeoff source selection strategy will have established evaluation 
criteria in the solicitation planning step to ensure that the government receives the best 
value based on cost and performance. Although the term best value is often incorrectly 
used as a synonym for the tradeoff source selection strategy, the FAR recognizes that best 
value can be obtained using an FPTA or tradeoff source selection strategy. It also 
recognizes that cost may not be the most important factor for award. Therefore, in 
situations where requirements are clearly defined and risks are low, the government 
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should adopt an LPTA source selection strategy based on cost/price to determine best 
value. However, in situations where requirements are not as clearly defined and risks are 
greater, cost becomes a less important factor for award (FAR 15.101). Here, we begin to 
see the importance of LPTA and tradeoff strategies as the government seeks the best 
value contractor. As indicated in Figure 3, the left side of the best value continuum shows 
price as the most important factor. The right side of the continuum shows non-cost 
factors, such as technology, as the most important. 


Non-cost factors 
less important 
than price 


Non-cost factors 
equal to price 


Non-cost factors 
more important 
than price 


Lowest price technically 
acceptable process 


Tradeoff process 


✓ 


When proposals are deemed 
technically acceptable, price is 
the determining factor 


Requirements clearly defined 
Development work low 
Contract performance risk low 


Importance of price decreasing 


Requirements less defined 
► Development work high 
Contract performance risk high 


Source GAO analysis of Federal Acqursition Regulation Part 15.101 | GAO-14-584 


Figure 3. Best Value Continuum. Source: GAO (2014). 


According to the LPTA source selection strategy, if the established technical 
standard is met, then the lowest offeror wins. This is represented on the left side of Figure 
3. Tradeoff selections occur when the government desires to achieve an optimal level of 
technical capability and past performance relative to cost/price in pursuit of a best value 
product or service. This process can range in complexity. As depicted in Figure 3, it can 
start moderately, and as it progresses to the right, price is seen to be less important than 
the advantage (non-cost factor) gained. 

3. Post-award Phase 

Contract administration and closeout are the two steps of the post-award phase 
and are the last steps in the contract management process (Garrett, 2010, p. 162). 
Contract administration is an important step within the post-award phase because it 
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ensures that all elements of the contract are adhered to. Contract closeout is also 
important because it settles all remaining contract requirements, deliverables, and 
payment. Due diligence in contract administration and closeout can help avoid surprises, 
especially with more complex contracts. 

a. Step 5: Contract Administration 

Contract administration is perfonned to ensure that both the contractor and 
the government adhere to the contract terms and conditions. It typically consists of 
“1) conducting a pre-performance conference, 2) monitoring the contractor’s work 
results, 3) measuring contractor’s performance, and 4) managing the contract control 
process” (Rendon & Snider, 2008, p. 176). The pre-performance conference brings all 
parties together at the beginning of the contract for clarifications of the contract. As part 
of step 5, quality assurance personnel monitor work results and keep the contracting 
officer apprised of their observations. Earned value management (EVM) is one tool that 
quality assurance personnel use to track progress in major systems acquisitions (FAR 
34.201). EVM measures planned cost and schedule against current status (Rendon & 
Snider, 2008, p. 178). For services, however, quality assurance is managed in accordance 
with the established quality assurance surveillance plan (QASP: FAR 37.604). The 
contract change control process is the portion of contract administration where changes 
are made to contracts as needed. When a change is needed, it must be made formally 
with contract modifications (Rendon & Snider, 2008, p. 178). There are two types of 
contract modifications. Bilateral modifications require the concurrence of both parties; 
unilateral changes can be executed by the contracting officer alone, as they are generally 
administrative in nature or do not fall outside the scope of the work (Rendon & Snider, 
2008, p. 178). If a unilateral change is desired, the contracting officer must issue a change 
order (Standard Form 30) in accordance with the changes clause to execute the action 
(FAR 43.201). 

b. Step 6: Contract Closeout and Termination 

Contract closeout or termination is the last step of the process in which the 
contract can either be closed out or tenninated for convenience or default (Rendon & 
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Snider, 2008, p. 180). All contracts must be closed in one of these three ways. Contract 
closeout, in a successful project, involves verifying project completeness and tending to 
administrative matters. The buyer has “inspected and accepted the supplies or services,” 
and any remaining invoices are processed (Garrett, 2010, p. 185). 

The other forms of closeout, tennination for convenience and termination for 
default, occur if successful completion cannot be achieved. When the government 
exercises a termination of the contract for convenience, it can be either complete or 
partial in accordance with the FAR (FAR 2.101). The government can do this 
unilaterally, without prejudice, to the contractor (Rendon & Snider, 2008, p. 180). 
Terminations for convenience require the government to “compensate the contractor 
fairly for the work done and the preparations made for the terminated portions of the 
contract, including a reasonable allowance for profit” (FAR 49.2). Termination for 
default is a different matter, and is the result of the government recognizing or 
anticipating failure by the contractor. If this measure is taken, it is also done in 
accordance with FAR Part 49. When the government decides to terminate a contract for 
default, it will not pay for “undelivered work and is entitled to repayment of advance and 
progress payments” (FAR 49.402). Using the default clause, the contracting officer can 
require that completed work and manufacturing materials associated with the job be 
transferred to the government. Before a tennination for default is finalized, the 
government will typically deliver a cure notice or a show-cause notice to the contractor 
(Snider & Rendon, 2008, p. 181). A cure notice simply informs the contractor of the 
problem and requires that the contractor make the correction. A show-cause notice 
requires the contractor to give reason why the contract should not be terminated. 

Additionally, during the contract closeout step, lessons learned are documented to 
help future project teams (Garrett, 2010, p. 188). CPARS ratings can be leveraged when 
documenting lessons learned (FAR 42.1501(b)). CPARS information is also an important 
factor in the source selection process, as we will see in the next section covering source 
selection strategies. 
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c. 


SOURCE SELECTION STRATEGY 


This section addresses in detail the topic of source selection strategies. Executing 
the appropriate source selection strategy is step four of the contract management process 
and includes the results of detailed market research and thorough requirement(s) analysis. 
Source selection is considered strategic in nature because the selected strategy may have 
a significant effect on contract outcomes. The two primary source selection strategies are 
LPTA and tradeoff. They are discussed in the following section. 

1. LPTA 

When the government detennines that cost/price is the most important evaluation 
criteria and that a minimum specified level of technical ability and past perfonnance will 
achieve best value, contracting officers will typically utilize an LPTA source selection 
strategy. More specifically, the lowest-priced bid or proposal will be selected if the 
contractor meets or exceeds a specified and acceptable level of technical ability and past 
performance. The LPTA source selection strategy dictates, “the evaluation factors and 
significant sub-factors that establish the requirements of acceptability shall be set forth in 
the solicitation” (PAR 15.101—2(b)( 1)). The source that meets or exceeds the standards 
specified by the government and has the lowest price will be selected as the provider. 
This source selection strategy does not permit tradeoffs when ranking proposals (Rene & 
Snider, 2008, p. 175). That said, the government can look at these but cannot use them to 
reject a source. Therefore, if a source meets the minimum standard and comes in at the 
lowest price, that source should win the award regardless of an alluring non-cost/price 
benefit from another source. Also, past performance will not be an evaluation criterion to 
differentiate contractors under the LPTA source selection strategy, assuming the 
contractors meet the minimum level of past performance as described in section 15.305 
of the PAR (PAR 15.101—2(b)(1)). 

2. Tradeoff 

When best value is achieved by seeking “other than the lowest priced offeror or 

other than the highest technically rated offeror,” the tradeoff strategy is preferred (PAR 

15.101-l(a)). This method allows the government some flexibility to award the contract 
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as shown in Figure 2 (Rendon & Snider, 2008, p. 175). This is done by allowing for 
“tradeoffs among cost or price and non-cost factors” which in turn “allows the 
Government to accept” a proposal that is not necessarily the lowest cost/price (FAR 
15.101). The solicitation must state two FAR requirements in order for this method to be 
valid. Best value, unlike LPT A, requires relative importance of evaluation factors to be 
clearly stated in the solicitation (FAR 15.101-1). All of the other evaluation factors 
(collectively) must be stated relative to cost/price in the solicitation. According to the 
FAR, the government should state, excluding cost/price as a factor, the importance of all 
the factors combined as being “significantly more important than, approximately equal 
to, or significantly less important than cost or price” in the solicitation (FAR 15.101). As 
previously stated, this gives the government flexibility in awarding the contract; however, 
if a higher-priced item is chosen, it must be justified and documented (FAR 15.101-1). 
Therefore, it is up to the contracting officer to know which criteria are most important 
when awarding the contract best value can be achieved. 

If these requirements are met, then the award should stand up to scrutiny. If not, a 
losing bidder could protest. The previous section discussed the contract management 
process and two different types of source selection strategies. The FAR states when each 
is appropriate. However, there is an ongoing debate as to which strategy actually provides 
the best value. The next section details recent GAO findings and the ongoing debate as to 
when contracting officers should employ the LPTA or tradeoff strategy, respectively. 

D. CURRENT DEBATE 

This section addresses relevant GAO findings and the current debate by 
practitioners within the DOD acquisition community as to when contracting officers 
should employ the LPTA or tradeoff strategy. These differing viewpoints on the subject 
indicate that the proper use of LPTA and tradeoff is still open to interpretation. 

Starting in 1992, the GAO detennined contract management in the DOD to be 
high risk (GAO, 2015). The GAO highlighted challenges in its investigative reports 
including acquisition workforce competency and capacity, aligning workforce with 
projected funding, service acquisition, contracting techniques and approaches, and 
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Operational Contracting Support (GAO, 2015). Of the aforementioned challenges, the 
areas of most concern with regard to source selection strategy are the contracting 
techniques and approaches. As a consequence of the DOD struggling with how to 
determine which contract type to utilize for a given requirement, the focus shifted to best 
value practices (GAO, 2015). As a result, Congress studied “DOD’s use of the best value 
Tradeoff process, specifically when non-cost factors were more important than price” 
(National Defense Authorization Act, 2010). Accordingly, contracts awarded in 2009 
valued at $25 million and up were reviewed concerning: (1) the frequency that the DOD 
utilized the best value tradeoff process by contract type, (2) DOD source selection 
strategy determinations and reasoning; and (3) challenges faced in the tradeoff process 
(GAO, 2010). The results of the review indicated that best value processes were used 
95% of the time for competitively awarded contracts. Figure 4 shows a breakdown of the 
different source selection strategies found by the GAO during its investigation; tradeoff 
source selection strategy was used in 69% of awarded contracts. 

Lowest price 
technically acceptable 

Sealed bid 0 

2% 

Non-cost factors less 
important than price 


Non-cost factors 
equal to price 


Non-cost factors more 
important than price 


| | Best value processes (95%) 

1 | Best value tradeoff process (69%) 

Source: GAO analysis of DOD contract documents 

Figure 4. Results of GAO Review by Source Selection Strategy. 

Source: GAO (2010). 
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The GAO found that when tradeoff source selection strategies were used, there 
was no difference between lower priced proposals and higher cost proposals in the DOD. 
When the DOD selected an offer that was not the lowest price, there was only a less than 
5% difference in price. However, not all cases were that low; more specifically, the 
purchase of burn-resistant clothing for marines in Iraq. The 48% cost differential 
outweighed the lower price of the next offeror’s proposal due to increased burn 
protection (GAO, 2010). The GAO concluded that for fiscal year 2009, the majority of 
competitive contract awards utilized best value tradeoff processes and that using that 
process effectively was dependent upon proper judgement in cost/price and non¬ 
cost/price evaluation factors. The GAO further suggested that the DOD create a viable 
training plan to aid their acquisition workforce in determining when a cost difference is 
justified during the tradeoff decision process (GAO, 2010). 

In 2014, the GAO also reviewed the DOD’s tradeoff process. The GAO examined 
contracts with obligations of over $1M, guidance on best value execution, and Defense 
Acquisition University (DAU)/ departmental training provided to acquisition 
professionals. As before, the GAO found that tradeoff source selection strategies were 
utilized in the majority of contracts reviewed. They found that LPT A was used primarily 
for low dollar commercial acquisitions. However, the use of tradeoff decreased by 11% 
and LPTA increased by 10% from years 2009-2013; theses changes are illustrated in 
Figure 5. 


Source selection 
process 

Fiscal year 2009 
percent 

Fiscal year 2013 
percent 

Statistically significant 
change 3 

Tradeoff 

69 

58 

Yes 

Lowest price technically 
acceptable 

26 

36 

Yes 

Sealed bid 

5 

6 

No 


Source; 2009: GAO-11-8; 2013: GAO analysis of DOD solicitation documents. | GAO-14-584 


Figure 5. Fiscal Years 2009 and 2013 Source Selections over $25 Million. 

Source: GAO (2014). 
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The GAO found that declining budgets caused the increase in use of LPT A source 
selection strategies. The GAO also found that organizations and DAU were providing 
more online and classroom training; however, DAU stressed that hands-on training was 
critical in ensuring truly educated decisions are made (GAO, 2014). 

From the seller’s viewpoint, Calisti describes some common misperceptions by 
industry concerning using the LPT A strategy and details when LPT A would be an 
appropriate source selection strategy and when it would not (Calisti, 2015, p. 17). Calisti 
describes industry’s concerns regarding the government’s perceived overuse of the LPTA 
source selection strategy (Calisti, 2015, p. 17). Industry believes the use of the LPTA 
source selection strategy results in “low cost, low quality” products and services, limits 
creativity and innovation, and reduces the DOD’s technological advantage (Calisti, 2015, 
p. 17). In this instance, the private sector is insinuating the LPTA source selection does 
not lead to successful contract outcomes. 

While Calisti generally agrees that the government often misuses the LPTA 
source selection strategy, he refutes industry’s opinion that the use of the LPTA source 
selection strategy only results in “low cost, low quality” products and services by 
describing when the use of LPTA is useful and appropriate (Calisti, 2015, p. 20). Calisti 
argues that the LPTA source selection strategy should be used when the government has 
a low-risk and a well-defined requirement and when cost/price is the most important 
evaluation criterion (Calisti, 2015, p. 20). Well-defined requirements detail all necessary 
product or service specifications in a manner that is understood by both the government 
and the contractor. Conversely, the tradeoff source selection strategy should be used 
when the government expects greater risk of unsuccessful contract performance and will 
accept a cost/price that is higher for greater technical performance (Calisti, 2015, p. 20). 
In summary, Calisti believes that how well defined the requirement is, coupled with the 
anticipated risk of the contract outcome, should be the determining factor in deciding 
whether to use an LPTA or tradeoff source selection strategy. 

Slate asserts that there are four relevant factors, regardless of source selection 

type, when awarding a contract. Those factors are “mission capability, proposal risk, past 

performance, and cost/price” (Slate, 2007, p. 17). This is in line with FAR 15.304. Slate 
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states that LPTA primarily focuses on acceptable mission capability over all other factors 
when detennining the lowest price offeror that will be awarded the contract (Slate, 2007, 
p. 17). This supports the overall tenor of the “technically acceptable” portion of the 
LPTA source selection strategy. When using the tradeoff source selection strategy, 
however, he says that all of the four previously mentioned factors are considered in order 
to award a contract (Slate, 2007, p. 18). Slate then asserts that this process demonstrates 
that the government might pay more for any or all of the four factors. This viewpoint is 
supported by the FAR (FAR 15.101-1). 

Bunting describes the perceived advantages and disadvantages of using LPTA or 
tradeoff strategies. Using an LPTA source selection strategy has advantages. These 
include “transparency,” “efficient source selection,” “low corruption risk,” and possibly 
being “protest proof’ (Bunting, 2014, p. 12). The appearance of transparency can reduce 
negative feedback and not have “a general perception of unfairness in the evaluation 
process because the award decision affords minimal discretion” (Bunting, 2014, p. 12). 
The LPTA source selection strategy is also more efficient than a tradeoff strategy. 
Essentially, only two questions will need to be asked regarding the proposal. Is the 
proposal technically acceptable, and if yes, is it the lowest priced (Bunting, 2014, p. 12)? 
This will allow the proposals to be differentiated using the two main evaluation factors 
and will lead to a quicker award decision. The next advantage is the possibility of low 
corruption risk. Purchaser discretion will be “limited to determining whether an offeror’s 
proposal meets the minimum standard of acceptability” (Bunting, 2014, p. 12). The final 
advantage of the LPTA source selection strategy is that it may be protest proof. This is 
because the LPTA source selection strategy involves “far less discretion and complexity 
than the tradeoff process” (Bunting, 2014, p. 12). 

The LPTA source selection strategy also has some disadvantages. These include 
“narrow application,” “high transaction costs,” “discouraged innovation,” and “reduced 
discretion” (Bunting, 2014, pp. 15-16). Since the appropriate use of an LPTA source 
selection strategy is for well-defined requirements, having a narrow application can lead 
to risks in awarding LPTA contracts that are not clearly defined (Bunting, 2014, p. 14). 
The second disadvantage is the potential for higher transaction costs. Contractors may 
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submit unrealistic cost proposals that meet the minimum technical specifications to win 
the contract award. In reality, the government may “incur significant financial costs in the 
long term for pursuing the lowest cost in the short-term” (Bunting, 2014, p. 15). To 
mitigate some of this risk, the FAR requires that contracting officers conduct cost realism 
analysis to determine if a contractor’s proposed cost/price is artificially low based on 
inconsistencies between the offeror’s technical and management approach and the 
associated cost/price (Garrett, 2010, p. 143). If the technical, management, and cost/price 
proposals are detennined to be inconsistent, the contracting officer may reject the 
contractor’s proposal as unacceptable. Alternatively, a cost competitiveness analysis can 
determine if a contractor’s proposed cost/price is too high compared to what is available 
in the market (Garrett, 2010, p. 143). If the proposed cost/price is determined to be too 
high, the proposal can be removed from the competitive range. The third disadvantage is 
the potential for discouraged innovation. The LPTA source selection strategy only 
requires the offerors to meet the minimum technical specifications and does not 
incentivize the contractor for producing higher standards. Innovations go beyond what is 
required; they do not influence the evaluation (Bunting, 2014, p. 15). Reduced discretion 
is the final disadvantage. The LPTA source selection strategy “limits the agency’s 
discretion to use of business judgment” and has a “reduced degree of flexibility” 
(Bunting, 2014, p. 16). 

Using the tradeoff source selection strategy has advantages compared to the 
LPTA source selection strategy. These include “learning,” “business judgment,” 
“flexibility,” and “customer satisfaction” (Bunting, 2014, pp. 5-7). The tradeoff strategy 
“provides an opportunity for the government to leam through the source selection 
process” (Bunting, 2014, p. 5). The government will be able to differentiate the quality 
differences that would otherwise only be known to industry experts (Bunting, 2014, p. 5). 
The next advantage is the use of business judgment. The tradeoff source selection 
strategy is highly subjective and this allows the source selection authority the ability to 
assess the qualitative measures based on their expertise (Bunting, 2014, p. 6). The third 
advantage is flexibility. The tradeoff process allows the source selection authority to have 
the flexibility to award the contract that is “not always the highest technically acceptable 
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rated proposal or lowest priced proposal” (Bunting, 2014, p. 7). The final advantage is 
customer satisfaction. By combining each tradeoff advantage, customer satisfaction can 
be improved by not limiting the contract award based only on cost/price factors (Bunting, 
2014, p. 7). 

The tradeoff source selection strategy also has some disadvantages. These include 
“corruption risk,” “suspicious offerors,” “game of chance,” and “complexity” (Bunting, 
2014, pp. 8-11). The first disadvantage is the risk of corruption. The source selection 
authority is afforded broad discretion and has the potential to abuse this discretion in 
contract awards (Bunting, 2014, p. 8). The next disadvantage is by having suspicious 
offerors. Since the tradeoff source selection strategy is subjective, there are common 
allegations of misusing discretion to include “favoritism, unequal treatment, undisclosed 
evaluation factors, wiring the specification to a particular offeror or unduly restrictive 
specifications, and bias or predetermining the awardee” (Bunting, 2014, p. 9). The third 
disadvantage is the game of chance. This occurs when the offeror “must correctly guess 
the agency’s preference for a high-quality, high cost solution or a lower-quality, lower- 
cost solution” (Bunting, 2014, p. 9). The final disadvantage is complexity. The tradeoff 
source selection strategy is extremely cumbersome and “requires extensive solicitation 
preparation, a lengthy evaluation period, and the business judgment to make sound 
tradeoff decisions,” and is “more susceptible to protests” (Bunting, 2014, p. 11). 

All of these articles revolve around the central theme of determining when the use 
of each strategy is appropriate. The disadvantages of each are even brought to the 
forefront by one of the authors. The articles stress the proper application of these source 
selection strategies. In doing so, an underlying question emerges that we will seek to 
answer with this research: Are pre-award and post-award contract elements affected by 
source selection strategy? If a relationship does exist, then acquisition professionals 
should devote more time and resources in detennining the appropriate source selection 
strategy. 
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E. SUMMARY 


Chapter II described applicable contract management statutes and regulations, the 
contract management process, and the differences between source selection strategies. It 
also detailed the current debate concerning LPTA and tradeoff source selection strategies. 
The information provided in this chapter lays the foundation and provides context for this 
research and analysis. Military systems commands are a valuable source of contract data 
to analyze how the LPTA and tradeoff source selection strategies affect short- and long¬ 
term contract outcomes. Chapter III will discuss the DOD acquisition organization and 
responsibilities, including two naval systems commands where the contract data was 
collected for this research. 
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III. NAVAL SYSTEMS COMMANDS 


This chapter discusses the two naval systems commands selected for this research, 
SPAWAR and NAVSUP, as well as the DOD acquisition chain of command to which 
they are included. More specifically, this chapter offers a top-level description of the 
acquisition organizational structure within the DOD and the Navy, as well as those of 
SPAWAR and NAVSUP. Although these two organizations have vastly different 
contracting requirements, the commonality is that they both follow applicable statutes, 
the FAR, and the six-step contract management process. The following analysis provides 
insight into the individual command organizational structures and responsibilities, 
mission sets and customers, and procurement and contract management departments. 

A. DOD ACQUISITION ORGANIZATION 

The acquisition organizational structure within the DOD is hierarchical in nature 
and comprises multiple echelons of expertise and support. The Undersecretary of Defense 
for Acquisition, Technology, and Logistics (USD[AT&L]) acts as “the principal staff 
assistant and advisor to the Secretary of Defense and Deputy Secretary Defense for all 
matters concerning acquisition, technology, and logistics” (OUSD[AT&L], 2015a). 
Figure 6 depicts the organizational structure of the Office of the USD[AT&L] 
(OUSD[AT&L], 2015b). This appointed official receives support from a principal deputy 
and oversees 11 unique directorates with responsibilities ranging from the analysis of 
acquisition resources to defense procurement and acquisition policy (OUSD[AT&L], 
2015b). The assistant secretary of defense for acquisition (OASD[A]) supports the 
individual directors by monitoring acquisition programs in areas such as tactical warfare 
systems; space; and strategic and intelligence systems; and C3, cyber, and business 
systems (OUSD[AT&L], 2015b). The professionals who fill these positions are 
acquisition subject matter experts and enable the USD[AT&L] to execute the 
responsibilities of supervising DOD-wide acquisition programs and establishing 
acquisition policies (OUSD[AT&L], 2015a). To achieve viable acquisition programs 
while protecting taxpayer funds, the OUSD[AT&L] guides DOD procurement and 
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contract management procedures by issuing three key policy documents including DOD 
Directive (DODD) 5000.01, DOD Instruction (DODI) 5000.02, and multiple iterations of 
the Better Buying Power initiative (DOD, n.d.-a). DODD 5000.01 “provides management 
principles and mandatory policies and procedures for managing all acquisition programs” 
and sanctions the publication of DODI 5000.02 (DOD, 2007). DODI 5000.02 provides 
detailed procedures to direct the operation of the DOD acquisition system in accordance 
with DODD 5000.01 (DOD, 2015). The BBP initiative was promulgated to “restore 
affordability in defense procurement and improve defense industry productivity” (DOD, 
n.d.-b). BBP expands on existing acquisition principles and procedures by concentrating 
on eight initiatives, as described in a memo from the Under Secretary of Defense: 

• Achieve affordable programs... 

• Achieve dominant capabilities while controlling lifecycle costs... 

• Incentivize productivity in industry and government... 

• Incentivize innovation in industry and government... 

• Eliminate unproductive processes and bureaucracy... 

• Promote effective competition... 

• Improve tradecraft in acquisition of services... 

• Improve the professionalism of the total acquisition workforce (Kendall, 

2015, Attachment 1) 

BBP applies to all components of the DOD acquisition organization from 
contracting officer’s representatives (CORs), contract specialists, and contracting 
officers, to program managers and senior acquisition executives. 
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Figure 6. AT&L Offices Organizational Chart. Source: OUSD(AT&L) (2015b). 

B. NAVY ACQUISITION ORGANIZATION 

Below the OUSD[AT&L], military service-specific acquisition executives 
implement contracting and program management policies in accordance with the DOD 
5000 series directives and instructions (ASN [RD&A] , n.d.-a). The assistant secretary of 
the Navy for research, development, and acquisition (ASN [RD&A]) “represents the 
Department of the Navy to USD(AT&L) and to Congress on all matters relating to 
acquisition policy and programs” (ASN [RD&A], n.d.-a). As portrayed in Figure 7, the 
ASN [RD&A]’s staff includes a principal military deputy and a principal civilian deputy, 
along with an assistant general counsel and a director for acquisition career management 
(ASN [RD&A], n.d.-a). ASN [RD&A] leads a large and complex naval acquisition 
organization composed of fourteen program executive offices (PEOs), seven systems 
commands (SYSCOMs), and nine deputy assistant secretaries (ASN [RD&A], n.d.-a). 
The fourteen PEOs and their program managers (PMs) supervise “the development and 
acquisition of Naval systems” including the Joint Strike Fighter, Space Systems, Littoral 
Combat Ships, and Unmanned Aviation and Strike Weapons, among others (ASN 
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[RD&A], n.d.-a). In coordination with the PEOs, the seven SYSCOMs and their various 
field offices directly manage the acquisition and support of the Navy’s complex weapons 
systems and capabilities (ASN [RD&A], n.d.-a). 


Office of the Assistant Secretary of the Navy 
(Research, Development and Acquisition) 



Figure 7. ASN [RD&A] Organization Chart. Source: ASN [RD&A], (n.d.-a). 


One of ASN [RDA]’s principal deputies is the deputy assistant secretary of the 
Navy for acquisition and procurement (DASN AP) (ASN [RD&A], (n.d.-a). DASN AP is 
the Navy’s primary subject matter expert regarding acquisition policy and the legal 
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framework including all applicable laws, statutes, and regulations (ASN [RD&A], n.d.- 
b). The DASN AP reports directly to ASN [RD&A] and is supported by a senior 
executive director and four subordinate directors (ASN [RD&A], n.d.-b). The DASN 
AP’s primary duties and responsibilities are to: 

• “Provide advice and staff support to the ASN(RD&A) on acquisition 

(Contracting & Business) and logistics issues 

• Serve as the DoN Competition Advocate General 

• Establish acquisition and logistics policy to: 

• Comply with laws and regulations 

• Promote & protect DoN & taxpayer interests 

• Ensure consistent and standardized business practices 

• Facilitate and improve the acquisition system” (ASN [RD&A], n.d.-b) 

One important aspect of the DASN AP’s policy responsibilities include the 
analysis and interpretation of the NMCARS, DFARS, and the FAR to ensure naval 
acquisition professionals receive the most accurate and up-to-date guidance (DASN AP, 
2016). 


C. SPA WAR ORGANIZATION 

The Space and Naval Warfare Systems Command (SPAWAR) enables and 
executes information warfare and dominance capabilities for the Department of the Navy 
(SPAWAR, 2016g). The tenn information dominance refers to “the operational 
advantage gained from fully integrating the Navy’s information functions, capabilities 
and resources to optimize decision making and maximize war-fighting effects” 
(Department of the Navy, 2013). SPAWAR is organized to develop, deliver, and sustain 
warfighter communications and information connectivity (SPAWAR, 2016g). SPAWAR 
consists of a headquarters office, two primary systems centers, and a space support 
activity. SPAWAR also maintains a partnership with a series of program executive 
offices (PEO) to acquire, deliver, and support the hardware and software needs of naval 
forces (SPAWAR, 2016g). PEO for Command, Control, Communications, Computers, 
and Intelligence (C4I) is tasked with acquiring, developing, and sustaining the Navy’s 
C4I capabilities (SPAWAR, 2016d). PEO for Space Systems manages the Navy’s space 
research, development, and acquisition activities (SPAWAR, 2016f). Lastly, PEO for 
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Enterprise Information Systems (EIS) manages the Navy’s information technology 
programs in support of all Sailors and Marines, both deployed and stateside (SPAWAR, 
2016e). Navy PEOs are echelon II organizations chartered by the Secretary of the Navy 
(SECNAV) (SPAWAR, 2016c). They work in consonance with the headquarters staff 
and the systems centers to create and sustain viable programs to support the warfighter. 
Figure 8 depicts SPAWAR’s organizational command structure (SPAWAR, 2016b). 



Figure 8. SPAWAR Organization Chart. Source: SPAWAR (2016b). 


SPAWAR is an echelon II organization led by a military flag-level commander. 
The commander executes the mission with the assistance of a military flag-level vice 
commander, a senior civilian executive director, a fleet readiness directorate, and a series 
of specialized departments (SPAWAR, 2016b). SPAWAR headquarters (SPAWAR HQ) 
organizes its departments numerically as follows: 

• “1.0 Comptroller 

• 2.0 Contracts 
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• 3.0 Office of Counsel 

• 4.0 Logistics and Fleet Support 

• 5.0 Chief Engineer 

• 6.0 Program Management 

• 7.0 Science and Technology 

• 8.0 Corporate Operations” (SPAWAR, 2016b) 

“SPAWAR is a competency aligned organization (CAO)” and arranges its 
departments to achieve and maintain the following eight core competencies: 

• Financial 

• Contracts 

• Legal 

• Logistics and Fleet Support 

• Engineering 

• Acquisition and Program Management 

• Science and Technology 

• Corporate operations (SPAWAR, 2016a) 

SPAWAR HQ 1.0 (comptroller) department executes the financial competency by 
providing program support services including cost estimation, business financial 
management, accounting, and budgeting (SPAWAR, 2016a). The comptroller staff 
coordinates with DOD budgeting offices and other internal departments to safeguard 
SPAWAR’s funding in a fiscally constrained environment. SPAWAR HQ 2.0 (contracts) 
department executes the contracting competency by performing procurement and contract 
management functions for SPAWAR’s program offices and related PEOs (SPAWAR, 
2016a). SPAWAR contracting officers and other contracting professionals leverage their 
business acumen and a cooperative relationship with industry to pursue best value 
acquisitions for the warfighter, the government, the industrial base, and the taxpayer 
while maintaining ethical standards of conduct. 

SPAWAR HQ 3.0 (office of counsel) department executes the legal competency 
by providing legal services with respect to acquisition statutes, intellectual property law, 
environmental regulations, standards of conduct, and FOIA programs (SPAWAR, 
2016a). SPAWAR’s legal professionals ensure command compliance with federal 
statutes and regulations, DOD policy directives, and internal guidance by providing legal 
advisory services. SPAWAR HQ 4.0 (logistics and fleet support) department executes 
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their competency by governing SPAWAR’s logistics and support efforts and ensuring 
that sound logistics principles are applied throughout the organization (SPAWAR, 
2016a). SPAWAR HQ 5.0 (chief engineer) department executes the engineering 
competency by providing technical leadership and systems engineering services for all of 
SPAWAR’s acquisition programs (SPAWAR, 2016a). 

SPAWAR HQ 6.0 (program management) department executes their competency 
by utilizing a common acquisition and project management framework to ensure major 
programs meet cost, schedule, and performance requirements (SPAWAR, 2016a). 
SPAWAR HQ 7.0 (science and technology) department executes their core competency 
by conducting scientific experimentation and leveraging industry and academia to 
identify and transform new technologies into useful applications (SPAWAR, 2016a). 
SPAWAR HQ 8.0 (corporate operations) department executes their core competency by 
supervising total force management, information technology management, corporate 
strategy, public affairs, and special program overview and compliance (SPAWAR, 
2016a). SPAWAR performs its diverse set of missions and core competencies through a 
geographically dispersed organizational structure. Figure 9 depicts the locations of 
SPAWAR offices and field activities throughout the United States (DTIC, 2012). 
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Figure 9. Primary and Subsidiary SPAWAR Offices. Source: DTIC (2012). 


SPAWAR HQ operates out of San Diego, California, along with SPAWAR 
Systems Center Pacific (SSC PAC), one of two echelon III systems centers within the 
SPAWAR hierarchy, and PEO C4I (DTIC, 2012). SSC PAC is the Navy’s primary 
RDT&E laboratory for command, control, communications, computers, intelligence, 
surveillance, and reconnaissance (C4ISR) and maintains field offices in Hawaii, Guam, 
and Japan (SPAWAR, 2016j). SPAWAR Systems Center Atlantic (SSC LANT) is the 
second echelon III systems center in the SPAWAR hierarchy. SSC LANT resides in 
Charleston, South Carolina, and maintains field offices in Norfolk, New Orleans, Tampa, 
and Washington, DC (SPAWAR, 2016i). SSC LANT also has a presence overseas 
including Stuttgart, Naples, Djibouti, and Bahrain (SPAWAR, 2016i). SSC LANT works 
in conjunction with PEO Space Systems, PEO EIS, and SPAWAR Space Field Activity 
(SSFA) to deliver multiple capabilities including C4ISR, information operations, 
information assurance, business information technology, enterprise information services, 
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and space systems (SPAWAR, 2016i). PEO Space Systems and PEO EIS are located in 
Virginia (DTIC, 2012). SSFA is an echelon III organization located in Northern Virginia 
and manages the Navy’s space sensors and satellite communications in conjunction with 
the National Reconnaissance Office (SPAWAR, 2016h). 

One commonality amongst all of SPAWAR’s echelon II and III organizations is a 
heavy reliance on contracting support. SPAWAR contracting professionals within 
SPAWAR HQ, SSC PAC, and SSC LANT work closely with industry to provide quality 
acquisition support to all program offices and SPAWAR PEOs. Figure 10 depicts 
SPAWAR’s contracting support in monetary terms during the span of one fiscal year 
(SPAWAR, 2015). 


/ SPAWAR \ 
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Purchased Services: Government 
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Figure 10. SPAWAR FY13 Macro Funds Flow. Source: SPAWAR (2015). 


According to SPAWAR’s latest macro funds flow analysis, the organization 

received $7.3 billion in funding for FY13 (SPAWAR, 2015). Of the $7.3 billion, 

SPAWAR HQ 2.0 (contracts) executed $2.9 billion in contract actions, with an additional 
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$2.7 billion and $1 billion executed by SSC LANT and SSC PAC contracting staffs, 
respectively, for a total of $6.6 billion (SPAWAR, 2015). SPAWAR’s contracting efforts 
directly support critical requirements, including the following: 

• Hardware and software development 

• Systems integration, assembly, and testing 

• Acquisition of contractor services 

• Research and development 

• Technical, industrial and logistics support (SPAWAR, 2015) 

These efforts are in line with SPAWAR’s primary mission and provide the 
warfighter with the greatest technical and communication capacity to operate effectively 
in any environment around the globe. 

D. NAVAL SUPPLY SYSTEMS COMMAND (NAVSUP) ORGANIZATION 

NAVSUP is an Echelon II command that provides quality of life support and 
global logistics support to naval and joint forces. Through a centralized inventory 
management system, they are able to provide material support to naval and joint weapons 
systems, ships, aircraft, and submarines. They also provide waterfront fleet logistics 
support, base operating support services, service and supply contracting, material 
delivery coordination, and provide material management and warehousing service 
solutions (NAVSUP, 20161). Figure 11 describes the organizational structure within 
NAVSUP. 
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Figure 11. Naval Supply Systems Command Organization. 

Source: NAVSUP Organization (2016k). 


Many of the Navy’s quality of life programs are managed by NAVSUP, including 
Navy Lodges, Naval Exchanges, Household Goods, and the Fleet Mail (NAVSUP, 
2016i). They manage the Navy’s Food Service Program and are responsible for 
generating and promulgating policy governing general mess operation at sea and ashore. 

Concerning contracting authority, NAVSUP Global Logistics Support (GLS) and 
Naval Exchange Command (NEXCOM) exercise unlimited contracting authority through 
the Navy Field Contracting System (NFCS) (NAVSUP, 2016f). NAVSUP Weapon 
Systems Support (WSS) is its own head of contracting activity (HCA) but NAVSUP code 
02 provides contract policy and oversight to headquarters and all NAVSUP elements. 
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Additionally, 1,200 purchase card programs fall under the NAVSUP HCA, two activities 
(NAVMEDLOGCOM and NAVOCEANO) fall under NAVSUP’s large contracting 
authority, and NAVSUP simplified acquisition procedures (SAP) authority is exercised at 
24 activities 

The NAVSUP Assistant Commander for Contracting (N7) strategically leads the 
NAVSUP contracting community and provides a structure for delivery of services 
throughout the Navy Field Contracting System (NFCS) (NAVSUP, 2016f). N7 also 
serves on the HCA principal staff for matters concerning contracting policy, review of 
operations, and approval actions. For purchasing matters or management of contracting 
matters, N7 acts for the HCA under purview of NAVSUP and executes policy and 
oversight for WSS. 

The NAVSUP Assistant Commander for Contracting also represents the HCA and 
NFCS on issues concerning higher authority. Working with the Inspector General (IG) 
and Office of General Counsel (OGC), N7 develops and issues procedural guidance to 
detect and combat fraud. N7 also functions as executive agent for matters concerning the 
Navy Supplies and Services Contingency Contracting Program, Navy Simplified 
Acquisition Program, Navy contract reporting, and Navy and Marine Corps Strategic 
Sourcing. In addition, N7 also functions as executive agent for all Navy contracting 
policy covering the Javits Wagner O’Day (JWOD) Act, the program manager for 
NAVSUP command Procurement Performance Management Assessment Program 
(PPMAP), and APC for NAVSUP overseeing all HCA purchase-card holders. In 
conjunction with NAVSUP Business Systems Center (BSC), N7 is also the functional 
lead for the Navy Electronic Commerce Online (NECO), issues with automated 
procurement, and the Standard Procurement System (SPS) (NAVSUP, 2016f). 

E. NAVSUP WEAPON SYSTEMS SUPPORT 

NAVSUP Weapon Systems Support (WSS) employs more than 2,300 military 
and civilians and serves as Navy and Marine Corps weapons systems supply chain 
manager (NAVSUP, 2016g). They provide program support to Naval, Joint and Allied 
forces weapons systems. 
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The 02 Contracts Directorate is employs over 200 military and civilians that 
operate as a unified organization from two geographically separated locations in 
Mechanicsburg and Philadelphia, Pennsylvania (Graham, 2013). Combined, more than 
34K contract actions valued at $3 billion are executed by the sites. 

NAVSUP WSS 02 consists of seven procurement departments with unique 
customer bases. They are responsible for a broad range of contracting services to include 
performance based logistics, services, repair, and procurement (Graham, 2013). 
Philadelphia houses three contracting departments; the departments provide support to 
the aviation arms of U.S. military services and foreign military. Mechanicsburg also 
houses three contracting departments, which provide support to maritime, foreign 
military operations, and information technology customers. The seventh department is 
located in both sites and acts as a resource for both sites that provides “acquisition policy, 
process management, training, and personnel management oversight across the entire 
directorate” (Graham, 2013). 

The seven departments mentioned above are primarily supplier-oriented; their 
primary goal is to build key relationships with suppliers in order to support cognizant 
weapons system programs throughout the acquisition life cycle (Graham, 2013). Each 
department is aligned with an Integrated Weapon Systems Team (IWST), which proves 
advantageous to NAVSUP in providing contracting and supply support for their 
customers. Additionally, two buying detachments from DLA are collocated within each 
of the major WSS sites in order to augment procurement of repairable and consumable 
spare parts. 

Some of the major acquisition programs include the following. 

1. Aviation - Philadelphia 

Code 022, Fixed Wing Contracts. “Provides contracting support to multiple IWST 
customers/platforms including F/A-18, AV-8, E-2/C-2, P-3, H-53/60/46, EA-6B and V- 
22” (Graham, 2013). A recent major contract was awarded for an aircraft auxiliary power 
unit. Utilizing a firm fixed price contract, the award is a five-year PBL renewal and 
serves as the foundation for future Joint Service/DLA PBL contracts (Graham, 2013). 
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Code 023, Rotary Wing Contracts. “Provides contracting support to multiple 
IWST customers/platforms, including the H-l, H-46, H-53, H-60, EA-6B, V-22, and E- 
2/C-2 (Graham, 2013).” A joint PBL was recently awarded for the V-22 aircraft, the 
contract will provide more than four years of support for over 168 items; valued at $218.4 
million, the cost-plus incentive fee contract will provide enduring support for the Air 
Force’s CV-22 and the Marine Corps’ MV-22 (Graham, 2013). 

Code 026, Common Systems. “Provides contracting support to all major aviation 
IWST customers and platforms” (Graham, 2013). Some recent major contract awards 
include a commercial firm fixed price PBL for the T700 turboshaft, providing three years 
of support. 1,500 Marine and Navy aircraft are supported by the PBL that is valued at 
more than $107 million (Graham, 2013). 

2. Maritime-Mechanicsburg 

Code 021, Maritime Systems. “Provides contracting support to multiple IWST 
customers and platforms, including Hull Mechanical and Electrical (HM&E), MK99 Fire 
Control, WSN-7, BPS, IBS, SPY Transmitter, Q-70, ARCI, MK41 VLS, USC-38, CEC, 
CIWS, and NATO Sea Sparrow” (Graham, 2013). A recent major contract award was for 
the AEGIS SPY-1 Radar, the PBL renewal is a five-year contract for AEGIS based fleet 
assets. Valued at over $37 million, the contract supports more than 1,650 items (Graham, 
2013). 

Code 024, Level 1/SUBSAFE and Ammunition Systems. “Provides specialized 
contracting support for CAD/PAD, Airborne Expendable Countermeasures Program, 
Lightweight Torpedoes, Level 1 Sub Safe, CARPER, TRIPER/AERP, and 2S COG 
(Graham, 2013).” Code 024 processes spare and repair requirements for the military 
sealift command and submarine fleet through requisitioning for replenishment of 
inventory and spot purchasing. “The Ammunition Systems division provides very 
specialized contracting support, primarily to external customers, for the procurement of 
flare and chaff, ammunition, pyrotechnics, and other munitions pertaining to training and 
operational requirements” (Graham, 2013). 
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Code 027, IT Products and Services. “Provides procurement support for 
automation hardware, software and systems from commercial sources for use by 
NAVSUP and other DoD operating sites” (Graham, 2013). Their infonnation technology 
services contracting office is the second largest in the DON and is responsible for over 
$400 million in procurement annually (Graham, 2013). 

3. Dual-Site 

Code 025, Acquisition Policy, Technology and Resources. “Provides all of the 
resource and budget management, policy dissemination, training, metrics and staff 
support for the entire Contracting Directorate... serves as NAVSUP’s program manager 
for the Integrated Technical Item Management and Procurement (ITIMP) contract writing 
system... performs internal contract reviews and is the lead for all contracting-specific 
reviews performed by external entities” (Graham, 2013). 

F. NAVSUP GLOBAL LOGISTICS SUPPORT 

NAVSUP Global Logistics Support (GLS) provides operational logistics 
capabilities to the Navy, Joint, and Allied Forces through eight subsidiary NAVSUP Fleet 
Logistics Centers (FLCs). Per the NAVSUP GLS website, they are based in San Diego; 
Norfolk, Virginia; Jacksonville, Florida; Yokosuka, Japan; Pearl Harbor, Hawaii; 
Bremerton (Puget Sound), Washington.; Sigonella, Italy; and Bahrain (NAVSUP, 2016e). 
NAVSUP GLS monitors the perfonnance of waterfront and support while managing 
operations of the NAVSUP FLCs. Each NAVSUP FLC offer the following support to 
their respective regions: contracting, global logistics services, hazardous material 
management, fuels, logistics operations, material management, regional transportation, 
integrated logistics support, household goods movement support, postal, warehousing, 
base supply support for Navy installations, and ammunition (NAVSUP, 2016e). 

For contracting, NAVSUP Global Logistics Support employs a lead contracting 
executive (LCE) who serves as contracting support lead for the NAVSUP FLCs. Five 
civilians and one military staff the LCE (NAVSUP, 2016f). The LCE’s mission is to 
manage all FLC field contracting operations as one streamlined entity operating across 
the globe (NAVSUP, 2016f). 
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The LCE is also responsible for field management over NAVSUP FLC 
contracting operations in conjunction with the Chiefs of Contracting in the following 
locations: San Diego, California; Norfolk, Virginia; Philadelphia, Pennsylvania; 
Bremerton, Washington; Jacksonville, Florida; Yokosuka, Japan; Sigonella, Italy; and 
Pearl Harbor, Hawaii (NAVSUP, 2016f). The FCE is also responsible for management 
over Procurement Perfonnance Management Assessment Programs (PPMAP) in Norfolk 
and San Diego, which administer and evaluate limited procurement authority purchase 
card activities and programs for NAVSUP proper (NAVSUP, 2016f). 

With an acquisition workforce spanning all of the FECs, GES employs over 
24,000 civilian employees and 345 military personnel. It also awards 85,000 to 88,000 
contracts worth between $3.6 billion and $4.4 billion annually (NAVSUP, 2016f). 

1. NAVSUP FLC San Diego 

NAVSUP FLC San Diego serves Navy, Marine, Coast Guard, and MSC fleet and 
shore commands/components throughout the Southwest region. According to the 
NAVSUP website, “NAVSUP FLC San Diego delivers combat capability through 
logistics by teaming with regional partners and customers to provide supply chain 
management, procurement, contracting and transportation services, technical and 
customer support, defense fuel products and worldwide movement of personal property” 
(NAVSUP, 2016c). 

Code 200, the contracting department, provides regional contracting support and 
is responsible for purchasing functions to include centralized buying as dictated by 
NAVSUP GLS based on NAVSUPSYSCOM guidance (NAVSUP, 2016d). They provide 
acquisition services for third party logistics, direct vendor delivery, A-76, base support 
services (household goods, food services, transportation, warehousing, and HAZMAT), 
engineering and technical support services, maintenance, equipment and hardware, port 
services, habitability, NMCI networks, and repair services and components for aircraft 
and ships (NAVSUP, 2016d). NAVSUP FLC San Diego Contracting Department is 
comprised of acquisition workforces located in San Diego, California; Seal Beach, 
California; Monterey, California; and Lemoore, California. 
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2. NAVSUP Business Systems Center 

NAVSUP Business Systems Center (BSC) supports the Navy, DOD components, 
and international partners with logistical support information technology. They 
accomplish this through executing design, development, sustainment, integration, and 
implementation of best-value logistics and financial business systems for the Navy, the 
DOD, international partners, and federal agencies (NAVSUP, 2016b). 

NAVSUP BSC functions as the central design agency for the Navy enterprise 
information technology and management solutions, emphasizing finance, business 
intelligence, and supply chain products and services (NAVSUP, 2016b). They are the 
primary agent for logistics and quality-of-life IT support, and they function as a bridge to 
other partners in the naval support network. As a central design facility, BSC is the 
DOD’s primary provider of non-tactical infonnation systems for the DOD, NAVSUP 
Enterprise, and international customers (NAVSUP, 2016b). 

3. Navy Exchange Service Command 

The Navy Exchange Service Command (NEXCOM) is based out of Virginia 
Beach, Virginia. Their primary mission is providing discounted goods and services to 
support quality of life programs for military active duty, reservists, retirees, and their 
dependents. NEXCOM is a field activity of NAVSUP (NAVSUP, 20161). 

NEXCOM oversees six major programs: 

• Navy Exchange (NEX) Retail Stores and Services 

• Ships Stores Program 

• Uniform Program Management Office 

• Navy Clothing and Textile Research Facility (NCTRF) 

• Navy Lodge Program 

• Telecommunications Program Office (NAVSUP, 20161) 

NEXCOM executes non-appropriated funds in support of its operations, the only 
exception being the Ship’s Store Program. As a result, NEXCOM is a self-sustaining 
organization and reinvests all profits from sales into Morale, Welfare, and Recreation 
(MWR) programs and NEX retail operations and equipment (NAVSUP, 20161). 
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Unlike civilian retail operations, the House Anned Services Total Force 
Subcommittee exercises congressional oversight over NEXCOM. Specialized laws 
“govern how military exchanges may operate in certain businesses or with certain types 
of merchandise” (NAVSUP, 2016). Regulation of alcohol and tobacco prices and a ban 
on the sale of adult periodicals are a few illustrations of congressional mandates in effect. 

G. REASON FOR SELECTING SPA WAR AND NAVSUP 

We selected SPA WAR and NAVSUP for our research project because they are 
major SYSCOMs with large procurement organizations that procure simple goods and 
services in addition to complex systems. SPAWAR executes a significant number of 
tradeoff type contracts, while NAVSUP commonly uses the LPTA source selection 
strategy due to their service-centric mission at the regional, fleet, and squadron support 
level. Between the two organizations’ multiple contracting source selections, we were 
able to source a sufficient combination of LPTA and tradeoff source selection strategies 
to address our research questions. 

H. SUMMARY 

Chapter III provided a synopsis of DOD acquisition infrastructure, primarily 
focusing upon the Navy’s acquisition activities and organization. It also included an 
overview of two major SYSCOMs, SPAWAR and NAVSUP; particularly their mission, 
organizational structure, and contracting divisions. Chapter IV will describe how we 
obtained the research data, the statistical analysis conducted on the data, the analysis 
findings, and the implications of the findings. 
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IV. DATA ANALYSIS 


Chapter IV conveys the research process and the data analysis employed in 
response to our research questions. This research is part of an ongoing research stream. 
Therefore, much of the language and techniques used to analyze the data are largely the 
same as previous research projects in order to maintain the integrity and consistency of 
the primary research body. The research stream also includes the works of Watson 
(2015), Lamoureux, Murrow, and Walls (2015), Bastola, Woodward, and Findley (2015), 
and Ban, Barnes, and Comer (2015). That said, all data used for this analysis is unique 
and was acquired from SPAWAR and NAVSUP; commands not previously used in this 
research stream. 

A. VARIABLES EXAMINED 

Our analysis contains two main dependent (outcome) variables (DVs) from the 
data collection categories detailed in the Appendix: PALT and CPARS ratings. PALT 
corresponds to the amount of time, in days, from the receipt of an approved requirements 
package to the date when the resulting contract is awarded. This DV is considered 
continuous for our analysis. CPARS ratings can be used as an indicator of positive or 
negative contract performance. These ratings are typically assigned periodically and at 
the conclusion of contract perfonnance. To assess contract performance, contractors are 
rated on the following variables: Cost, Quality, Schedule, Business Relationship, and 
Subcontracting. Likert categorization has been assigned for CPARS ratings. Therefore: 

• 5=Excellent 

• 4=Very Good 

• 3=Satisfactory 

• 2=Marginal 

• l=Unsatisfactory 

In this research, we used an overall average of CPARS ratings as the second DV 
(i.e., we combined individual CPARS ratings to make an average CPARS rating for each 
contract). 
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In this analysis, the DV or outcome may vary from the influence of a single 
independent variable (IV): source selection strategy. More specifically, the IV refers to 
the LPTA and tradeoff source selection strategies. We designated the IV as LPTATO, 
where a 0 represents LPTA and a 1 represents tradeoff. 

This data analysis uses five covariate variables from the data collection categories 
listed in the Appendix. These five items can potentially influence the association between 
the IV and DV. The five covariates used in this data analysis are continuous variables and 
are as follows: (1) value of the contract in dollars (VALUE), (2) number of evaluation 
factors (NUMEVALFACTORS), (3) number of reviews (NUMREVIEWS), (4) number 
of offers (NUMOFFERS), and (5) number of contract line item numbers (NUMCLINS). 
It is our wish to parcel out any covariate effects so that the true relationship between the 
IV and DVs become more visible. 

B. DESCRIPTIVE STATISTICS 

Table 1 displays the descriptive statistics that resulted from our analysis and 
breaks down each variable into data subcategories. The subcategories are sum of all data 
(top), sum of LPTA data (middle), and sum of tradeoff data (bottom). 
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Table 1. Descriptive Statistics 


Descriptive Statistics 

Variable 


Obs 

Mean 

Standard 

Deviation 

Min 

Max 

PALT [days) 

Sum of all Data 

Sum of LPTA 

Sum of Tradeoff 

29 

245.8966 

237.60220 

14 

990 

18 

126.1111 

81.24682 

14 

370 

11 

441.9091 

280.64440 

84 

990 

Average 

CPARS 

Rating 

Sum of all Data 

Sum of LPTA 

Sum of Tradeoff 

19 

3.528070 

0.6784617 

2.2 

5 

5 

3.600000 

0.5477226 

3.0 

4 

6 

3.325000 

0.9293815 

2.2 

5 

Value 

Sum of all Data 

Sum of LPTA 

Sum of Tradeoff 

43 

$18,700,000.00 

$22,500,000.00 

$27,819.07 

$92,600,000.00 

18 

$1,368,585.00 

$1,380,352.00 

$27,819.07 

$4,499,432.00 

13 

$30,200,000.00 

$28,000,000.00 

$99,999.43 

$92,600,000.00 

Number of 

Evaluation 

Factors 

Sum of all Data 

Sum of LPTA 

Sum of Tradeoff 

30 

2.633333 

1.0662000 

1 

5 

17 

2.058824 

0.6586528 

1 

3 

13 

3.384615 

1.0439080 

2 

5 

Number of 

Reviews 

Sum of all Data 

Sum of LPTA 

Sum of Tradeoff 

30 

7.366667 

7.049007 

2 

25 

18 

4.444444 

5.260533 

2 

25 

12 

11.750000 

7.300374 

2 

24 

Number of 

Offers 

Sum of all Data 

Sum of LPTA 

Sum of Tradeoff 

42 

3.404762 

3.298629 

1 

12 

18 

3.833333 

4.514682 

1 

12 

13 

3.692308 

2.056883 

1 

8 

Number of 

CLINs 

Sum of all Data 

Sum of LPTA 

Sum of Tradeoff 

31 

17.35484 

51.52834 

1 

290 

18 

22.00000 

67.09782 

2 

290 

13 

10.92308 

13.51827 

1 
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C. DATA CONCERNS 

The data extracted from contracts that used the LPT A source selection strategy 
consist of 18 PALT and five CPARS ratings. Likewise, for those using the tradeoff 
source selection strategy, 11 PALT and six CPARS ratings were extracted. Power 
calculations (a = .05, [3 = .80) suggest we need five PALT cases and 41 CPARS cases to 
achieve adequate statistical power. In this case, the required amount of PALT cases was 
collected. However, we were unable to collect enough CPARS cases to achieve adequate 
statistical power. The latter point should be kept in mind when interpreting the results. 1 
There is also a small disproportionality in the data pertaining to the amount of cases for 
the strategies examined. The assignment of sum squares can become problematic when 
there is uncertainty about the mean in this unbalanced design. Regardless of these 
concerns, solutions exist. Instead of utilizing a grand mean, 2 we can apply a weighted 


1 A total of 65 cases (i.e., contract files) were collected; however, not all cases had sufficient data to be 
included in the analysis. 

2 In a balanced design, the grand mean acts as an intercept. 
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mean. In addition, the Stata 12.1 statistics software used for our analysis perfonns 
assignment of sums of squares. 

D. RESEARCH PROCESS 

Our focus is to investigate variations among contract outcomes based on source 
selection strategy. In accordance with statistical practice, a methodology based on group 
differences is required. Therefore, source selection strategies (LPTA and tradeoff) must 
be divided into respective groups in an effort to detennine if contract outcomes (PALT 
and CPARS ratings) differ between the groups. 

Our plan originally was to use multivariate analysis of covariance (MANCOVA) 
to evaluate the group differences. MANCOVA evaluates “statistical differences on 
multiple continuous dependent variables by an independent grouping variable, while 
controlling for a third variable called the covariate” (Statistics Solutions, n.d.). The data, 
however, failed to meet the assumptions required for MANCOVA (specifically, they 
failed the linearity assumption between the DVs), so we were forced to examine the DVs 
separately, using analysis of covariance (ANCOVA), which is a univariate method. 

The ensuing issues are addressed by ANCOVA: After considering covariate 
effects, what is the likelihood that mean differences among the groups on a given DV 
occurred by chance? Once the effect(s) of the covariates (VALUE, 
NUMEVALFACTORS, NUMREVIEWS, NUMOFFERS, and NUMCLINS) have been 
parceled out, is there a significant difference between the PALT mean value in 
procurements that use LPTA as opposed to the PALT mean value in procurements that 
use tradeoff? Similarly, is there a significant difference between the mean value of 
CPARS ratings in procurements that uses LPTA as opposed to the mean value for 
CPARS ratings in procurements that use tradeoff once the effect of the covariates have 
been removed? 

E. ASSUMPTION TESTING 

We tested some assumptions about the data prior to performing the ANCOVA. 
The tests were as follows: 
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First, Mahalanobis’ Distance and scatter plots were used to find outliers. We 
found five outliers and chose to drop those observations from subsequent analyses, as 
outliers are known to significantly affect ANCOVA (Tabachnick & Fidell, 2007, 
pp. 200-203). 

Second, normality of the sampling distribution was evaluated by examining 
density graphs to assess skewness and kurtosis. While normality is assumed in situations 
“with relatively equal sample sizes in groups, no outliers, and two-tailed tests ... [and] 20 
degrees of freedom for error” (Tabachnick & Fidell, 2007, p. 202), we do not meet the 
degrees of freedom criterion with our CPARS data. The density graphs indicated that the 
dependent variables were not normally distributed. To achieve a normal distribution, the 
dependent variables and covariates were logarithmically transformed. 

Third, linearity was evaluated by studying 40 different scatter plots. The variables 
used in the scatter plots included all pairs of covariates and all pairs of DV-covariate 
combinations for each source selection strategy. The scatter plot analyses indicated that 
many of the DV-covariate relationships were non-linear. We chose to drop those 
covariates because “[cjovariates are often included as a convenience in reducing error, 
but it is hardly a convenience if it [sic] reduces power” (Tabachnick & Fidell, 2007, 
p.251). After dropping the troublesome covariates, the remaining covariates are 
NUMRE VIEWS and NUMOFFERS for the PALT DV, and VALUE and 
NUMEVALFACTORS for the CPARS DV. 

Fourth, analysis of variance (ANOVA) was used to examine regression 
homogeneity. These ANOVAs consist of the IV, each of the covariates, and an 
interaction term between the IV and each of the covariates. The interaction tenns were 
not significant, which indicates that the relationship between the DVs (PALT or CPARS 
ratings) and their associated covariates is the same at both levels of the IV (LPTA or 
tradeoff). Therefore, the assumption of regression homogeneity is met. 

Fifth, an assessment of multicollinearity was performed by evaluating the squared 
multiple correlation for each covariate. Multicollinearity is not an issue. 
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Sixth, homogeneity of covariance between groups was evaluated using the 
multivariate test of means provided in Stata 12.1 statistics software. This test checks 
whether population variances and the covariance of both DVs are equal for each of the IV 
groups. All grouping cells were homogenous. 

Seventh, the reliability of the covariates was examined. Because all the covariates 
are factual calculations of actual contract data, we have no reason to suspect unreliability. 
With the data properly tested and prepared, the ANCOVAs were performed. 

F. RESULTS 

1. PALT 

The results suggest that source selection strategy, with PALT as the DV, yields 
substantial variations in time-to-contract, even after adjusting for the covariates 
(NUMREVIEWS and NUMOFFERS) (F(3,25) = 7.13, p <. 01). Neither of the covariates 
is statistically significant. The strength of the relationship was moderate, with rf = .22. 
The results indicate that tradeoff source selections take 105% longer than LPTA source 
selections (b 3 = 1.05, p < .01). This result supports the anecdotal evidence that tradeoff 
source selections take more time than LPTA source selections. Table 2 and Table 3 
represent the results of ANCOVA and regression analysis, respectively, for the PALT 
DV. 


Table 2. PALT ANCOVA Output 


Number of Observations = 29 
Root MSE = 0.722369 
R-squared = 0.4610 
Adj. R-squared = 0.3963 


Source 

Partial Sum of Squares 

Degrees of Freedom 

Mean Square 

F-Distribution 

Prob > F 

Model 

11.1559193 

3 

3.71863977 

7.13 

0.0013 

Innumreviews 

0.483079791 

1 

0.483079791 

0.93 

0.3452 

Innumoffers 

0.013554514 

1 

0.013554514 

0.03 

0.8733 

LPTATOl 

5.24428376 

1 

5.24428376 

10.05 

0.004 

Residual 

13.0454237 

25 

0.52181695 


Total 

24.2013431 

28 

0.86433368 
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Table 3. PALT Regression Output 


Number of Observations = 29 
F(3,25) = 7.13 
Prob > F = 0.0013 
R-squared = 0.4610 
Adj. R-squared = 0.3963 
Root MSE = 0.72237 


Source 

Sum of Squares 

Degrees of Freedom 

Mean Square 

Model 

11.1559193 

3 

3.71863977 

Residual 

13.0454237 

25 

0.52181695 

Total 

24.2013431 

28 

0.86433368 


Inpaltreq 

Coefficient 

Standard Error 

t 

P> |t| 

95% Confidence Interval 

Innumreviews 

0.1880682 

0.1954632 

0.96 

0.345 

- 0.2144958 

0.5906321 

Innumoffers 

0.0274795 

0.1705006 

0.16 

0.873 

- 0.323673 

0.378632 

LPTATOl 

1.052352 

0.3319532 

3.17 

0.004 

0.3686812 

1.736022 

_cons 

4.385241 

0.2934172 

14.95 

0 

3.780936 

4.989545 


2. CPARS Ratings 

The results suggest that source selection strategy, with CPARS ratings as the DV 
and VALUE and NUMEVALFACTORS as the covariates, does not yield substantially 
dissimilar CPARS ratings, (F(3,6) = 0.43, ns). Neither of the covariates is statistically 
significant. It is important to note the small sample size for this ANCOVA (required n = 
41). With only 10 observations, there is not enough power to detect statistical 
significance. More data are required to confirm these results. Table 4 and Table 5 
represent the results of ANCOVA and regression analysis, respectively, for the CPARS 
DV. 


Table 4. CPARS ANCOVA Output 


Number of Observations = 10 
Root MSE = 0.253161 
R-squared = 0.1768 
Adj. R-squared = -0.2348 


Source 

Partial Sum of Squares 

Degrees of Freedom 

Mean Square 

F-Distribution 

Prob > F 

Model 

0.082590764 

3 

0.027530255 

0.43 

0.7394 

Invalue 

0.031062576 

1 

0.031062576 

0.48 

0.5124 

numevalfact 

0.007897081 

1 

0.007897081 

0.12 

0.7376 

LPTATOl 

0.065499193 

1 

0.065499193 

1.02 

0.3511 

Residual 

0.384543186 

6 

0.064090531 


Total 

0.46713395 

9 

0.051903772 
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Table 5. CPARS Regression Output 


Number of Observations = 10 
F(3,6) = 0.43 
Prob > F = 0.7394 
R-squared = 0.1768 
Adj. R-squared = -0.2348 
Root MSE = 0.25316 


Source 

Sum of Squares 

Degrees of Freedom 

Mean Square 

Model 

0.082590764 

3 

0.027530255 

Residual 

0.384543186 

6 

0.064090531 

Total 

0.46713395 

9 

0.051903772 


InCPARS 

Coefficient 

Standard Error 

t 

P> |t| 

95% Confidence Interval 

Invalue 

0.0339787 

0.0488073 

0.7 

0.512 

- 0.0854484 

0.1534057 

numevalfact 

- 0.036401 

0.1036997 

- 0.35 

0.738 

- 0.2901451 

0.217343 

LPTATOl 

- 0.1726467 

0.1707801 

- 1.01 

0.351 

- 0.5905305 

0.2452371 

_cons 

0.9175872 

0.6970996 

1.32 

0.236 

- 0.7881542 

2.623329 


3. Kruskal-Wallis H Test 

Finally, because the CPARS variable was measured using Likert scores and some 
contend that Likert scores are ordinal rather than interval, we performed the Kruskal- 
Wallis H test. It is a “rank-based nonparametric test that can be used to determine if there 
are statistically significant differences between two or more groups of an independent 
variable on a continuous or ordinal dependent variable” (Laerd Statistics, 2013). 
Examining CPARS ratings as an ordinal variable, the results show no difference in 
CPARS ratings based on source selection method (x (1) = -533, p = .47), which confirms 
the results we found in the ANCOVA; however, this test also suffers from lack of 
statistical power. The box plot in Figure 12 displays these results. 
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Box Plot of Average CPARS Rating by Source Selection Method 
To Accompany the Kruskal-Wallis H Test 
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Figure 12. Box Plot of Average CPARS Rating by Source Selection Method 

G. SUMMARY 

Chapter IV described the results of the statistical analysis performed using the 
data from SPAWAR and NAVSUP. It provided a synopsis of the variables examined, a 
brief overview of the descriptive statistics utilized, identified data concerns, discussed the 
methodology used, and provided outcomes of the data analysis. Chapter V provides an 
overview of our analysis, the conclusions that were reached, and additional research 
topics to be examined in future studies. 
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V. SUMMARY, CONCLUSIONS, AND AREAS FOR 
FURTHER RESEARCH 


A. SUMMARY 

The goal of federal contracting is to obtain best value in all procurements, 
especially given the constrained budgetary environment. The contract management 
process provides the framework for achieving best value for the government and the 
taxpayer. The six steps of the contract management process are comprised of 
procurement planning, solicitation planning, solicitation, source selection, contract 
administration, and contract closeout. The source selection strategy is identified in the 
solicitation planning step and is executed in the source selection step of the process. 
Contracting professionals use either the LPTA or the tradeoff source selection strategy to 
achieve best value. However, there is a debate as to when the use of an LPTA or tradeoff 
source selection strategy is appropriate. Much of the debate concerns the potential 
relationship between contract outcomes and the strategy utilized. Relying on anecdotal 
evidence, many believe that LPTA source selections typically result in lower quality 
supplies and services and incentivize contractors to submit proposals below their actual 
costs in order to earn the contract award and then increase total contract cost/price 
through post-award modifications. Conversely, anecdotal evidence suggests tradeoff 
source selections take more time to award (PALT) than LPTA source selections because 
of the (typically) more extensive evaluation of non-cost/price factors. Moreover, 
anecdotal evidence suggests that tradeoff source selections result in a greater contract 
cost/price and have a higher risk of protest. To test these anecdotes, this research sought 
evidence of a statistical relationship between source selection strategy and contract 
outcomes, to include PALT and CPARS ratings. Our team extracted data from contracts 
at SPAWAR HQ, SSCPAC, and NAVSUP FLC San Diego to support this ongoing 
research stream. Various statistical models were used to determine if answers exist to the 
following research questions: 

(1) Are pre-award contract elements affected by source selection strategy? 

Pre-award elements include amendments to solicitation, how long it takes 
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to award the contract (PALT), contract type, and whether there was a 
protest. 

(2) Are post-award contract elements affected by source selection strategy? 
Post-award elements include CPARS ratings, and, if applicable, EVM 
metrics. 

(3) Are there any patterns or trends based on federal supply codes when 
LPTA or tradeoff is used as a source selection strategy at different systems 
commands? 

B. CONCLUSIONS 

Based on the results of our statistical analysis, the answers to our research 
questions are as follows: 

(1) Are pre-award contract elements affected by source selection strategy? 
Pre-award elements include amendments to solicitation, how long it takes 
to award the contract (PALT), contract type, and whether there was a 
protest. 

The results suggest that there is a statistically significant relationship between 
source selection strategy and PALT. More specifically, contracts awarded using the 
tradeoff source selection strategy took, on average, 105% longer to award than contracts 
awarded via the LPTA strategy. There was not sufficient data to support a statistically 
significant relationship between source selection strategy and the number of amendments 
to a solicitation. Similarly, there was not enough data to conclude that a statistically 
significant relationship exists between source selection strategy and the likelihood of a 
protest taking place. We arrived at the same conclusion regarding the relationship to 
contract type. 

(2) Are post-award contract elements affected by source selection strategy? 
Post-award elements include CPARS ratings, and, if applicable, EVM 
metrics. 

The results indicate that there is not enough data to confirm if a statistically 
significant relationship exists between source selection strategy and CPARS ratings. For 
contracts reviewed that used the LPTA source selection strategy, five sets of CPARS 
ratings were extracted. Likewise, for the contracts that used the tradeoff source selection 
strategy, seven sets of CPARS ratings were extracted. Power calculations suggest we 
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need 41 CPARS cases to achieve adequate statistical power. As a result, we were unable 
to collect enough CPARS cases to determine if a statistically significant relationship 
exists. Similarly, a statistically significant relationship could not be determined between 
source selection strategy and EVM metrics due to a lack of available data. 

(3) Are there any patterns or trends based on federal supply codes when 
LPTA or tradeoff is used as a source selection strategy at different systems 
commands? 

Statistical analysis indicates that there is not enough data to confirm if a 
significant relationship exists between source selection strategy and federal supply codes 
at different systems commands. More data are required to complete this analysis. 

C. AREAS FOR FUTURE ANAUYSIS 

Based on our results, the following areas were identified for future analysis. These 
areas are based on data gaps, which may provide additional insight into the relationship 
between source selection strategy and contract outcomes. 

1. EYM and CPARS Data 

Previous iterations of this research stream have been unable to accumulate 
sufficient EVM and CPARS data to determine if a statistically significant relationship 
exists between source selection strategy and contract outcomes. During our research, we 
were only able to collect 10 CPARS data observations, and we did not reach the 41 
observations required to provide adequate statistical power. During the initial 
coordination with the supporting commands, greater emphasis should be made in 
selecting contracts with associated CPARS information. Similarly, we were unable to 
find any EVM data within the contract files reviewed at SPAWAR and NAVSUP. Future 
research teams should access major weapon system contract data that includes EVM data 
and analyze accordingly. As more empirical data are obtained, additional analysis can be 
conducted to see if the relationships between source selection strategy and contract 
outcomes differ for various types of requirements (aircraft, ships, submarines, space 
assets, etc.). 
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2. 


Effect of New DOD Source Selection Procedures 


The latest DOD Source Selection Procedures (DFARS PGI 215.3, dated March 
31, 2016) incorporates the most recent best practices identified in pre-award peer reviews 
and program manager focus group reviews (DOD, 2016, p. 1). One area for further 
research is to ascertain if implementation of the new DOD Source Selection Procedures 
affects future source selection strategy detennination. Research teams should collect 
post-implementation source selection trend data to assess the document’s impact on the 
relationship between source selection strategy and contract outcomes. Statistical analysis 
should be conducted to compare pre- and post-implementation data to identify any 
significant trends and the implications thereof. 
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APPENDIX. DATA COLLECTION CATEGORIES 


Data Collection Catagories 

Basic Info 

Acquisition Complexity 

Observation 

# 

Predominant 

Contract 

Type 

# Evaluation 

Factors 

Eval Factor 

#4 

Life Cycle 
Stage 

# 

Competitive 

Range 

Incentive Fee 

Criterion #5 

Service 

SB Set Aside 

Eval Factor 

#1 

# Subfactors 

in EF#4 

Joint Urgent 
Operational 
Need (JUON) 

Single or 
Multiple 
Award 

Award Fee 

Criterion #1 

Organization 

Consolidated 

Requirement 

# Subfactors 

in EF#1 

Eval Factor 

#5 

# CLINs 

On-Ramp 

Provision 

Award Fee 

Criterion #2 

Contract 

Number 

Multiyear 

Eval Factor 

#2 

# Subfactors 

in EF#5 

Pre- 

Solicitation 
Conference/ 
Industry Day 

Incentive Fee 

Criterion #1 

Award Fee 

Criterion #3 

Name 

# Years 

#Subf actors 

in EF#2 

# 

Subelements 

Pre-Proposal 

Conference 

Incentive Fee 

Criterion #2 

Award Fee 

Criterion #4 

NAICS 

Dollar Value 

Eval Factor 

#3 

ACAT/SCAT 

Level 

Pre- 

Award/Pre- 

Performance 

Conference 

Incentive Fee 

Criterion #3 

Award Fee 

Criterion #5 

PSC 

Trade Off or 

LPTA 

# Subfactors 

in EF#3 

FAR Part 

# Offerors 

Incentive Fee 

Criterion #4 
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Data Collection Catagories 

Environmental Factor 

Outcome Variables 

Other 

It Reviews 

#Team Locations 

PALT- 

Requirement 
Receipt Date 

Protest 

EVM CV 

RFP 

Amendment 

Reason 2 

# Evaluation 

Notices 

Review 1 Date 
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